1. Field of Invention
The present invention relates generally to the field of non-visual displays. More specifically, the present invention is related to the non-visual display of multi-part information pages.
2. Discussion of Prior Art
Visually impaired users of computers implementing graphical user interface (GULI) environments face a number of issues. These issues arise because separate content is often displayed simultaneously in different spatially located areas of the screen in these environments, while the interfaces for visually impaired users represent this content in a linearized fashion. The problems arising from displaying content in a spatially distributed manner are particularly troublesome for visually impaired users who attempt to interface with the World Wide Web.
Many pages on the World Wide Web utilize xe2x80x9cframes,xe2x80x9d which partition the information displayed into a number of discrete segments in the user""s browser in order to create complex navigation tools for the site. When the user follows a link in one of the frames, the corresponding new content typically appears in either the frame that contained the originating link or in a single different frame, but the other frames in the display usually remain unchanged. Frames pose a major difficulty to visually impaired users because, after clicking a link, it is not immediately obvious to them that only one of several frames was updated.
This change is not immediately obvious to visually impaired users because visually impaired users typically use screen readers to access the content of Web pages. These screen readers linearize the sequence of items in a layout based on the hierarchical structure of the page. As a consequence, even though only one frame has been updated, it is necessary to scan through the entire page layout and re-discover the page as a whole to determine where changes occurred.
In addition, as these methods to render the output to a visually impaired user do not provide an overall layout of the areas in which content is displayed, there is not a common frame of reference for collaboration between a sighted user and visually impaired user. The visually impaired user receives a linearized version of a page, while sighted users can perceive the layout of a page more or less as a whole. When a sighted and a visually impaired user work together on a Web page, this difference in how to perceive the layout of the page can make it quite difficult to refer to items on the screen. The concept of xe2x80x9cin the top right corner of the screenxe2x80x9d is not directly usable for the visually impaired user. This impairs any collaboration effort between these two types of users.
The following references are examples of the recent efforts to provide better interfaces for visually impaired users to access information, or to provide non-visual feedback to a user generally. However, whatever the precise merits, features and advantages of the references below, none of them achieve or fulfills the purposes of the present invention.
U.S. Pat. Nos. 6,088,675 and 6,085,161 describe a system and method for auditorially representing pages encoded in a mark-up language. These references teach the presentation of the content of a Web page as a linear stream of audio information, while avoiding the visual metaphor of a screen. By avoiding the visual metaphor of a screen, a common frame of reference between sighted and visually impaired users cannot be provided.
U.S. Pat. No. 6,078,308 describes the use of force feedback in a graphical user interface environment to provide force feedback click surfaces. These feedback click surfaces are buttons, icons and other specific graphical objects. When a cursor controlled by the user moves over the surface of these specific objects, force feedback slows the movement of the input device the user is utilizing to control the cursor. By continuing to move the cursor past the feedback click surface, the object is selected, as if the user had xe2x80x9cclickedxe2x80x9d on it with a traditional mouse type input device. This non-visual force feedback, however, is not utilized to represent a higher-level abstract of the information displayed to allow a visually impaired user to understand and follow the overall layout of the visual representation of the information.
U.S. Pat. No. 6,125,385 describes the use of force feedback to provide effects to objects in a Web page. One such effect described is an xe2x80x9cenclosurexe2x80x9d effect, where a force is placed upon the input device to force the cursor into a geometrically bound area on the screen, or prevent the cursor from entering a geometrically bound area on the screen. Disclosed is the association of enclosure effects with frames of a Web page having particular objects of interest. Further disclosed is the use of a proxy server to pre-process a Web page to add force feedback effects to the Web page, prior to the page being received at the client. A specific teaching, however, of this force feedback to provide a non-visual representation of the overall layout of the visual Web page for a visually impaired user is not provided. In addition, providing information concerning the content of a frame using different non-visual feedback is not disclosed.
U.S. Pat. Nos. 5,825,308; 5,956,484; and 6,101,530 also describe the association of force feedback with graphical objects of a graphical user interface or Web page, such as an enclosure effect to attract the cursor to an object by controlling an input device. U.S. Pat. No. 5,825,308 additionally discloses the use of physical forces on the input device to increase a user""s perceptual understanding of the GUI spatial landscape portrayed on a display screen. None of these references, however, describe the use of different, non-visual feedback to provide further information about the object, or about any content displayed within the object. In addition, they do not specifically teach the use of force feedback to provide a common frame of reference for sighted and visually impaired users examining visually displayed data.
U.S. Pat. No. 5,736,978 describes the use of a tactile stimulator array to provide feedback to a non-sighted user to represent a visually displayed graph. As the stimulator array is moved across a digitizer pad, pins in the tactile array are raised to stimulate the user""s finger in areas corresponding to the lines of the visually displayed graph. In this manner, the displayed content itself is represented, i.e. the lines of the graphical information, as an illusion of raised features appearing underneath a user""s fingers. Audio feedback is utilized to provide additional information about this graphical data, such as axis labels, graph names, titles. While this reference describes the use of two types of non-visual feedback, it teaches the use of these two types to represent various aspects of the non-partitioned content itself. This contrasts with the present invention, which provides one type of non-visual feedback to indicate spatial relationships between different displayed areas of separate content, while a second type of non-visual feedback indicates meta-information about the content in an area.
U.S. Pat. No. 5,977,867 is of general background interest in the sense that it describes an input device of a type that is similar to that preferably utilized in conjunction with the present invention.
The literature xe2x80x9cMethod for Providing Position Relative Audio Feedback in a Scrollable Content Area,xe2x80x9d IBM Research Disclosure, v42 n418 p 225, February 1999, describes the use of audio feedback based upon the current size and position of scrollable content within a window being displayed. Disclosed is the use of different sounds to indicate different frames, where the sounds indicate the size of the content within a given frame and the user""s relative position within the content. The use of non-visual feedback to represent the page in a higher-level abstract representation, showing major sections in their approximate spatial relationships (e.g., top left, bottom center), however, is not described.
The literature, xe2x80x9cCooperative Web Navigation System for Blind and Sighted People,xe2x80x9d IBM TDB vol. 4 No. 11 p. 35, November 1997, describes a method to allow a sighted person and a blind person to share the same computer, either to do entirely separate work or cooperatively navigate the Web. Disclosed is the use of different input and output devices for a sighted user and blind user. A mouse and display is used for a sighted user, while a speaker and numeric pad are used for a blind person. However, no mechanism is described to allow a visually impaired user to understand and follow the overall layout of the visual representation of an information page.
A system and method is provided for non-visually displaying a multi-part information page containing different spatially located areas of separate content. To display the multi-part information page, a non-visual, abstract representation of the multi-part information page is maintained. This non-visual, abstract representation has non-visual display coordinates, which include boundary coordinates and content coordinates. The boundary coordinates define boundaries between the different spatially located areas and are associated with tactile feedback. The content coordinates define the different spatially located areas. Each of the content coordinates is associated with auditory feedback representative of content meta-information for the content of that area. Input position coordinates received from an input device are mapped to the non-visual display coordinates. When position input coordinates map to the boundary coordinates, a tactile output is generated to cause a tactile output device to generate the associated tactile feedback. When position input coordinates map to the content coordinates an auditory output is generated. The auditory output causes an auditory output device to generate the auditory feedback associated with the content coordinates.